Field of the Invention
The invention relates to a method for optimizing occlusion, and more particularly to a method for optimizing occlusion in augmented reality based on depth cameras.
Related Art
Recently, depth sensing technologies have been successfully applied to various industries. For example, the entertainment industry has developed novel man-machine interaction functions based on human body tracking and motion recognition enabled by depth cameras. A common principle of depth sensing is to emit infrared light to the target scene and thus to generate laser speckle patterns on the objects in the scene. Then, the laser speckle data is obtained through a sensing lens and the depth values are calculated. When the infrared light is projected onto an object in the scene, errors may be generated in the laser speckles due to various reasons such as occlusion or poor reflective properties on the object surface. The depth value of the object will be incorrectly estimated. This problem is likely to occur around the object edges. In addition, random noises of the laser speckles sometimes cause inconsistency or information loss of the depth data. As a result, the depth image taken by depth cameras becomes unstable and jittering.
The augmented reality (AR) technology combines virtual objects with the real world, and thus provides interactions between them to enhance user experience. The depth information of the real scene captured by depth cameras helps position virtual objects in augmented reality environment. Quality visualization of the content mixed with real and virtual objects requires proper processing of their mutual occlusion. In order to display virtual objects correctly, a common method is to directly perform a depth buffer (Z-buffer) calculation according to the depth data and the current viewing angle of the camera. The occluded portion of the virtual objects will not be displayed. However, the effectiveness of this method highly depends on the data quality provided by the depth cameras. The Z-buffer method tends to produce incorrect occlusion result with depth data of poor quality. For instance, some depth data pixels are lost, failing to occlude the virtual object properly. The object edges or silhouettes occluded often become rugged. Moreover, the data inconsistency between frames causes jittering of the occluded edges or silhouettes. These problems significantly deteriorate the visualization quality in augmented reality applications.